Skip to content

litellm_fix_mapped_tests_core: disable aiohttp transport in tests#20194

Closed
shin-bot-litellm wants to merge 1 commit intomainfrom
shin/fix-mapped-tests-core
Closed

litellm_fix_mapped_tests_core: disable aiohttp transport in tests#20194
shin-bot-litellm wants to merge 1 commit intomainfrom
shin/fix-mapped-tests-core

Conversation

@shin-bot-litellm
Copy link
Contributor

Problem

Tests using @respx.mock were hitting real APIs because respx only intercepts httpx requests, but litellm uses aiohttp transport by default.

Tests affected:

  • test_send_email_missing_api_key
  • test_send_email_multiple_recipients (resend & sendgrid)
  • test_search_uses_registry_credentials
  • test_vector_store_create_with_simple_provider_name
  • test_image_edit_merges_headers_and_extra_headers

Solution

  1. Add disable_aiohttp_transport fixture (autouse=True) to test files that use respx mocking. This forces httpx transport so respx can intercept requests.
  2. Clear client cache in fixture to prevent reuse of old clients.
  3. Fix isinstance checks in vector store tests to use type name comparison (avoids module identity issues from sys.path.insert).

Regression

PR #19829 (commit f95572e3ed) added @respx.mock but did not account for aiohttp transport bypassing respx interception.

Testing

  • These tests should now pass in CI without hitting real APIs
  • The disable_aiohttp_transport fixture ensures respx can mock all HTTP calls

## Problem
Tests using @respx.mock were hitting real APIs because respx only
intercepts httpx requests, but litellm uses aiohttp transport by default.

Tests affected:
- test_send_email_missing_api_key
- test_send_email_multiple_recipients (resend & sendgrid)
- test_search_uses_registry_credentials
- test_vector_store_create_with_simple_provider_name
- test_image_edit_merges_headers_and_extra_headers

## Solution
1. Add disable_aiohttp_transport fixture (autouse=True) to test files
   that use respx mocking. This forces httpx transport so respx works.
2. Clear client cache in fixture to prevent reuse of old clients.
3. Fix isinstance checks in vector store tests to use type name
   comparison (avoids module identity issues from sys.path.insert).

## Regression
PR #19829 (commit f95572e) added @respx.mock but didn't account
for aiohttp transport bypassing respx interception.
@vercel
Copy link

vercel bot commented Jan 31, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Error Error Jan 31, 2026 10:59pm

Request Review

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Jan 31, 2026

Greptile Overview

Greptile Summary

Changes Overview

This PR fixes a test reliability issue where tests using @respx.mock were making real API calls instead of using mocked responses. The root cause was that litellm uses aiohttp transport by default, which bypasses respx's httpx-only interception.

What Changed

  1. Added disable_aiohttp_transport fixture: Five test files now include an autouse fixture that:

    • Sets litellm.disable_aiohttp_transport = True to force httpx transport
    • Clears the client cache via in_memory_llm_clients_cache.flush_cache() to prevent reuse of old aiohttp clients
    • Properly restores the original state after tests complete
  2. Fixed isinstance checks in vector store tests: Replaced direct isinstance checks with type name comparisons (type(obj).__name__ == "ClassName") to avoid module identity issues caused by sys.path.insert in test setup.

Technical Analysis

The fix is well-designed and addresses the regression from PR #19829. By forcing httpx transport, respx can now properly intercept HTTP calls in these test files:

  • test_resend_email.py - Email sending tests
  • test_sendgrid_email.py - Email sending tests
  • test_main.py - Image edit header merging test
  • test_vector_store_registry.py - Vector store registry credential tests
  • test_vector_store_create_provider_logic.py - Vector store provider logic tests

The fixture implementation follows best practices by:

  • Using getattr with None default for safe attribute access
  • Properly restoring original values or deleting the attribute if it didn't exist
  • Clearing the client cache to ensure fresh clients are created with the new transport setting

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are isolated to test files only, properly implement cleanup/restoration logic, and directly address the identified regression. The fixture pattern is consistent across all modified files and follows pytest best practices.
  • No files require special attention

Important Files Changed

Filename Overview
tests/test_litellm/enterprise/enterprise_callbacks/send_emails/test_resend_email.py Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking
tests/test_litellm/enterprise/enterprise_callbacks/send_emails/test_sendgrid_email.py Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking
tests/test_litellm/test_main.py Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking
tests/test_litellm/vector_stores/test_vector_store_create_provider_logic.py Replaces isinstance checks with type name comparison to avoid module identity issues from sys.path.insert
tests/test_litellm/vector_stores/test_vector_store_registry.py Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking

Sequence Diagram

sequenceDiagram
    participant Test as Test Execution
    participant Fixture as disable_aiohttp_transport
    participant LiteLLM as litellm module
    participant Cache as in_memory_llm_clients_cache
    participant Respx as @respx.mock
    participant HTTPX as httpx transport
    
    Note over Test,HTTPX: Before PR (Broken)
    Test->>Respx: Start test with @respx.mock
    Respx->>LiteLLM: Intercept HTTP calls
    LiteLLM->>LiteLLM: Use aiohttp transport (default)
    Note over Respx,LiteLLM: respx cannot intercept aiohttp!
    LiteLLM-->>Test: Real API call made ❌
    
    Note over Test,HTTPX: After PR (Fixed)
    Test->>Fixture: autouse=True fixture runs
    Fixture->>LiteLLM: Set disable_aiohttp_transport = True
    Fixture->>Cache: flush_cache() to clear old clients
    Cache-->>Fixture: Cache cleared
    Fixture->>Test: Setup complete
    
    Test->>Respx: Start test with @respx.mock
    Respx->>LiteLLM: Intercept HTTP calls
    LiteLLM->>HTTPX: Use httpx transport (forced)
    Respx->>HTTPX: Mock HTTP request
    HTTPX-->>Respx: Return mocked response
    Respx-->>Test: Test passes with mock ✓
    
    Test->>Fixture: Test complete, teardown
    Fixture->>LiteLLM: Restore original disable_aiohttp_transport
    Fixture-->>Test: Cleanup complete
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@shin-bot-litellm shin-bot-litellm deleted the shin/fix-mapped-tests-core branch January 31, 2026 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants